DOCUMENT RESUME 



ED 358 129 



TM 019 911 



AUTHOR 
TITLE 

SPONS AGENCY 
PUB DATE 
CONTRACT 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Kultn, Gerald 

A Theory of Classroom Assessment and Teacher Practice 
in Mathemat ice . 

Department of Education, Washington, DC. 

Apr 93 

R117E10152 

19p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (Atlanta, 
GA, April 12-16, 1993). 

Reports - Evaluative/Feasibility (142) — 
Speeches/Conference Papers (150) 

MF01/PC01 Plus Postage. 

Classroom Observation Techniques; Classroom Research; 
^Educational Assessment; Educational Change; 
Educational Practices; Educational Theories; 
^Elementary School Teachers; Elementary Secondaxy 
Education; Knowledge Level; ^Mathematics Instruction; 
Mathematics Teachers; ^Secondary School Teachers; 
Student Evaluation; ^Teaching Methods; ^Theory 
Practice Relationship; Thinking Skills 
^Alternative Assessment 



ABSTRACT 

The theoretical and practice-based framework is 
presented for a study that explored the effect of teachers 1 use of 
alternative assessment on their mathematics teaching. . A model for the 
impact of alternative assessment on teacher knowledge and classroom 
teaching processes is proposed, building on work by E. Fennema and M. 
L. Franke (1992) and P. L. Peterson (1988). A research study to test 
the model is described, including a description of the coursework 
provided and a discussion of the methodology and data sources. 
Subjects were 17 female teachers and 1 male teacher (4 African 
Americans and 14 Whites) in grades 4 through 10. In addition to data 
from observations of teachers, results from informal data are 
presented, including class notes, teacher-made tests, and summaries 
written by the teachers describing assessment strategies. These data, 
along with those from observations and student reports, provide 
evidence that knowledge and use of alternative assessments did help 
mathematics teachers change their teaching strategies to reflect 
approaches shown by research to result in higher order thinking. One 
table gives a sample course syllabus, and one figure illustrates 
teacher concepts. (Authot/SLD) 
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ABSRACT 



This paper presents the theoretical and practice-based framework 
for a study which explored the effect of teachers' use of alternative 
assessment on their mathematics teaching. A model for the impact of 
alternative assessment on teacher knowledge and classroom teaching 
processes is proposed, building on work by Fennema & Frank (1992) and 
Peterson (1988). A research study to test the model is described, including 
a description of the coursework provided, and discussion of the 
methodology and data sources. Finally, the paper presents results from 
informal data that was collected, including class notes, teacher-made tests, 
and summaries written by the teachers describing assessment strategies 
that worked or did not work in their classes. These data, along with those 
from teacher observations and student reports, provide evidence that 
knowledge and use of alternative assessments did help mathematics 
teachers to change their teaching strategies to reflect approaches shown by 
research to result in higher order thinking. 
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A Theory of Classroom Assessment and Teacher Practice in Mathematics 
Gerald Kulm, Texas A&M University 

This study focused on the need to help teachers learn to improve the 
way they evaluate students' mathematics learning. The major objective 
was to develop assessment approaches that more accurately reflect 
teachers' own goals, the goals of the NCTM Curriculum and Evaluation 
Standards (1989), and other national recommendations. A more indirect, 
but important objective of the project was to study the impact of 
alternative assessment on classroom teaching practice. 

Recent experience in American mathematics education seems to 
verify that the material tested is the material taught. The evidence from 
the National Assessment of Educational Progress (NAEP), for example, has 
shown that computational skills have been the focus for competency tests, 
spawning textbooks and instructional emphases aimed at developing these 
skills in students. Teachers have been legitimately concerned that if they 
"fight the system" and teach higher order thinking, their students would 
suffer on the computationally oriented tests that they are required to pass. 
Many educators believe that very little change will occur in mathematics 
curriculum and teaching without a concurrent change in testing, especially 
in state and national standardized tests that are used to assess and 
compare student, school, and district performances (Kulm, 1990). 

Mathematics educators have called for new approaches to 
assessment and provided some directions for changes (Charles & Silver, 
1989; Kulm, 1990; NCTM, 1989; Romberg, 1990). Promising work on 
developing new approaches to mathematics assessment has begun to 
appear at the national, state, and local level (CCSSO, 1989; NAEP, 1987; 
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Pandey, 1990; Stenmark, 1989; Strong, 1990). However, little research has 
been done on specific approaches to mathematics assessment and how 
they work with various student populations. 

Most mathematics teachers believe that higher order thinking is 
important. In the Second International Mathematics Study, more than 60 
percent of U.S. mathematics teachers listed their highest goal as 
"developing a systematic approach to solving problems and developing an 
awareness of the importance of mathematical everyday life" (Crosswhite et 
al., 1986). Student performance on the IMS and recent (NAEP) tests 
indicate, however, that the aspirations of teachers and the performance of 
their students are very different things. Apparently, teachers are unable to 
accomplish what they would like to be able to do. 

Why don't mathematics teachers reach their own goals for teaching 
higher order thinking? There are several possibilities, including: (a) 
teachers really are trying to teach higher order thinking, but the students 
cannot learn, (b) teachers think that they are teaching higher order content 
but are actually teaching rote application of methods to solve special 
classes of word problems, (c) state and local tests demand that teachers do 
other things, or (d) teachers don't know how to evaluate student progress 
and achievement of higher order thinking. All of these explanations are 
probabl; true to some extent for many teachers. However, the last issue of 
how to evaluate higher order mathematical thinking would go a long ways 
toward overcoming other barriers. Successful teaching of anything, 
including higher order thinking in mathematics, is dependent upon the 
ability to determine the degree to which it has been learned. Valid and 
usable tests can provide an impetus for teaching higher order skills. 
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In this project, the goal was to bring instruction and assessment into 
closer alignment and interaction. This was made possible by helping 
teachers to develop the ability to assess the effects of teaching approaches 
that were designed to involve students in problem solving and thinking. 
Teachers were encouraged to broaden their repertoire of teaching 
strategies and to use alternative assessment strategies as a part of 
implementing these strategies. Work on developing alternative 
assessment plans contributed to bringing teaching and testing into 
agreement. Instead of being limited by traditional tests, teachers 
developed plans for assessment that fit within their own styles and 
approaches. 

Teacher Change Model 

Research on the effects of alternative assessment on the activities in 
teachers classrooms was a key objective of the project. We were 
interested to determine whether the use of alternative assessment had an 
effect on teachers' classroom instruction and on their students 1 attitudes 
toward mathematics. Fennema & Franke (1992) proposed a model which 
hypothesizes that teachers' knowledge develops in context, through the 
dynamic and interactive impact of knowledge of mathematics, pedagogical 
knowledge, and knowledge of learners' cognitions in mathematics. We 
propose that alternative assessments enhance teachers' knowledge in all 
three of these areas, but especially their knowledge of learners' cognitions 
and their knowledge about effective teaching. This knowledge enables 
teachers to transform their content knowledge into powerful pedagogical 
forms (Schulman, 1987). 
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In order to observe the outcomes of teachers' empowered 
pedagogical knowledge, we must look at those aspects of the classroom 
context which reflect powerful learning. A review of research on teaching 
by Peterson (1988) suggested that three primary classroom processes are 
essential for promoting the learning of higher-order thinking: (1) an 
emphasis on meaning and understanding, (2) encouragement of student 
autonomy and persistence, and (3) direct teaching of higher order thinking 
strategies. The interplay of these factors is depicted by Figure 1. 

|- Emphasis on meaning & understanding] 
■Direct teaching of thinking strategies] 




Knowledge 
of 

Mathematics 



Context 
Specific 
Knowledge 



Pedagogical 
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Figure 1. Teachers' Knowledge: Impact of Alternative Assessment 



In the proposed model, knowledge about alternative assessment 
strategies act to enrich teachers' knowledge of learners' cognitions. 
Teachers learn about their students' mathematical knowledge, 
understanding, and problem solving processes. They also leam how 
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students work on extended and complex problems, and how they 
communicate their work through oral, written, graphic, and other 
approaches. Finally, they learn how students do mathematics in individual 
and group situations. This enriched and in-depth knowledge about 
students has an impact on teachers' pedagogical knowledge. Teaching 
strategies that were dormant or not well developed become activated and 
intensified. More traditional teacher-centered approaches are used less 
often in light of assessments about their effectiveness in meeting students' 
cognitive needs. In some instances, knowledge of alternative assessment 
may also act upon and extend teachers' mathematical knowledge through 
work in developing tasks and evaluating students' mathematical 
performances. 

Procedures 

In order to test the proposed model, the project consisted of three 
coordinated components designed to help teachers plan and implement 
alternative assess, nent approaches: (1) Graduate course work on 
mathematics assessment to build knowledge about new assessment 
approaches and support implementation efforts, (2) Teacher development 
of assessment items and implementation of plans and strategies for 
student assessment, and (3) Development of assessment tasks and 
materials to support implementation efforts by teachers. Each of these 
components will be described in this chapter. 

Selection of Participants: We tried to obtain a diverse group of teachers 
who worked in rural, suburban, and urban areas, who taught students of 
different backgrounds and ethnicities, and who had a wide range of 
abilities r.id experiences as teachers. Eighteen teachers, ranging from 
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fourth to tenth grades, from school districts in east central Texas were 
selected for participation in the project. The teachers were a very diverse 
and able group. Their experience ranged from 3 to 20 years of teaching. 
All of them had some graduate work, 7 of them had completed Master's 
degrees. There were 17 women and one man; four African Americans, the 
rest White. 

Graduate Course Work : The teachers enrolled in and completed a 3 -credit 
graduate course on mathematics assessment. The course will met twice a 
month, throughout the school year at Texas A&M. The outline for the 
course is shown in Table 1. 



Course Syllabus for EDCI 689 Innovative Mathematics Assessment 



EDCI 689. Innovative Mathematics Assessment. Credit 3 hours. 
Course D escription : 

This course provides concepts, information, and strategies for designing 
multiple approaches to assessing mathematics performance, with special 
emphasis on helping teachers to develop comprehensive assessment plans, 
and providing support for classroom implementation. 
Content Outline : 

Class Content 



Table 1 



Session 



1 



National and state mathematics assessment standards 



2 



Theories of mathematics learning and assessment 
Types of mathematical learning to be assessed 
Overview of new mathematics assessment strategies 
Student mathematical products and outcomes 
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Portfolios of student mathematical performance 


7 


Assessment of minority and at-risk students 


8 


Assessment of individual student work 


9 


Assessment of cooperative learning groups 


10 


Development of specific assessment plans and methods 


1 1 


Assessment of student attitudes and beliefs 


1 2 


Holistic Qcnrino nihripQ anH cfrrQt*»oi**e 


13 


Assessment and grading 


14 


Summary and evaluation of strategies 


Evaluation: 





Text readings and assignments 25% 
Assessment Plan 50% 
Final Paper 25% 
Textbooks : 

Kulm, G. (1990). Assessing higher order thinking in mathematics. 
Washington, DC: American Association for the Advancement of 
Science. 

Stenmark, J. K. (1989). Assessment alternatives in mathematics. Berkeley, 

CA: University of California. 
Vermont State Board of Education. (1990). Looking beyond the answer: 

Report of Vermont's mathematics portfolio assessment program. 

The focus of the course was on an analysis of current approaches to 
authentic mathematics assessment such as those outlined in the texts and 
assigned readings. In addition, special attention was given to assessment 
models developed by several state assessments such as California, 
Connecticut, and Vermont. As a required course project, each teacher 



developed a plan for implementing alternative assessment in one of their 
mathematics classes or courses. The teachers also participated in 
developing and pilot testing a set of prototype assessment tasks. (See the 
section on the Alternative Assessment Question Bank for more information 
on this activity). 

Assessment Plans an d Implementation: Shortly after the beginning of the 
course, each teacher was asked to develop a tentative plan for 
implementing alternative assessment strategies. They were asked to 
identify one class in which to do this work. For elementary teachers, of 
course, this was their only class. The junior and senior high school 
teachers chose the class they felt most comfortable working with, one in 
which they thought the content was most compatible with alternative 
assessment, or one in which they felt students had the greatest need for 
change or motivation. There was a wide range of reasons for making these 
selections, and the teachers were not forced or directed to make any 
particular choice. 

There was a great deal of early discussion about the types of changes 
in assessment to make and the schedule for doing so. The teachers were 
encouraged to make gradual changes and to involve their students, and 
possibly parents, in plans or discussions of how changes would be made. 
Teachers were also encouraged to try and work toward using alternative 
assessment for a majority of testing and evaluation in the selected class by 
the end of the year. 

During the course, the teachers implemented their assessment plans 
and shared experiences on what worked in their classes. Much of the 
discussion at class meetings was devoted not only to providing formation 
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on assessment, but to encouraging the teachers to share and clarify 
approaches that they were trying in their classes. Among the issues that 
arose consistently were: 

- approaches to assigning grades 

- the use of scoring protocols on open ended items and problems 

- integration of writing, such as journals, into the evaluation process 

- the use of timed tests 

- the assessment of group vs individual work during cooperative learning 
activities 

- the attitudes of students in accepting new assessment approaches 

As classroom implementation took place, it became clear that much 
cf the discussion about procedures and strategies focused on teaching as 
much as on assessment. Many of the teachers' plans for implementing 
alternative assessment approaches were closely tied to strategies for using 
problem solving or communications activities as a regular part of 
instruction. For example, several teachers implemented a "problem of the 
week or problem of the day" approach to help students develop higher 
order thinking abilities. Some of these teachers used the problems at 
times as cooperative group activities; others asked students to select their 
best problem for presentation or evaluation; others asked students to 
choose a problem to include in a portfolio. 

Several teachers implemented group and project assignments, 
working with hands-on materials or calculators. Although many of the 
participants were comfortable with these kinds of strategies, the project 
and the input from fellow participants encouraged a few teachers to try 
these methods either for the first time or to make them a regular part of 
their instruction. 



Data Collection 

Classroom Observations: A classroom observation instrument was 
developed, focusing on the three processes essential for promoting the 
learning of higher-order thinking. Special attention was given to 
characterizing teaching approaches that reflect non-traditional strategies 
such as problem-based instruction, small group and other student- 
centered work, and attention to student learning as opposed to "covering" 
mathematics content. The following paragraphs describe the general 
procedures used to collect observation data. 

An experienced doctoral student in mathematics education observed 
the teachers, using the observation instrument to rate the extent to which 
the teacher exhibited each of the behaviors. An initial set of observations 
was done early in tb~, project, prior to teachers* implementation of their 
assessment strategies. This first observation served to establish baseline 
data on teaching strategies. During the school year, each of the teachers 
was observed two more times, once near the middle of the project, and 
once at the end of the school year. These observations provided data that 
reflected changes in teaching approaches during the year. 

In addition to collecting teacher behavior ratings from the observer, 
each teacher also rated themselves on each of the observation items, 
following the lesson. These ratings were discussed with the teacher as part 
of the post-lesson interview session. This self-rating served to focus 
teachers' attention on the types of teaching strategies that are identified as 
ones that enhance higher-order thinking skills in students. The self-rating 
data also provided a source for validating the observation ratings. 
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Finally, each observation was video-taped. Although this video- 
taping was not a part of the original proposal, we decided that it would 
provide additional valuable data. The complete analyses of the video- 
tapes are not reported here, but will become a focus of the dissertation by 
the doctoral student. We did use the video-tapes as a source for doing a 
reliability analysis of the observation ratings. 

Journals. Classroom Tests, and Quizes- Each teacher kept a journal of their 
classroom work and activities, focusing on assessment implementation. 
The journals were handed in at each class meeting. There were no specific 
guidelines for the journals, but teachers were encouraged to record their 
approaches to assessment, how they worked, reflections on special areas of 
interest, good ideas, special classroom episodes, or any other information 
that would reveal the progress of implementing their assessment plans. 
Many of the journals described the types of assignments and activities that 
were used to assess student learning, and the reactions of both themselves 
and their students as they made ? F iese changes. 

Samples of classroom tests used by the teachers were collected at the 
end of the project. These items were intended to provide indicators of how 
extensively the teachers actually implemented the innovative assessment 
approaches. Of special interest was the extent to which the new 
assessment approaches begin to permeate the day-to-day classroom 
quizzes and tests, and whether the classroom tests show variety in their 
format and setting. 

Student Questionnaires; Student questionnaires provided another indicator 
of whether teachers changed their teaching and assessment approaches. 
On the same schedule as the ob lations at the beginning of the year, the 
middle, and again near the end, students were given a 25-item 
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questionnaire. The first ten items provided an indication of whether the 
innovative testing approaches had an impact on the way students viewed 
mathematics. The last IS questions provided indicators, from the students 1 
point of view, whether teachers 1 classroom practices changed as a result of 
the assessment work. 

Summary of Teacher Outcomes 

In this project, we have made some significant progress in 
developing teachers 1 knowledge of how to assess problem solving and 
other types of higher order thinking. This knowledge appears to provide 
teachers with the freedom to choose and implement instructional activities 
which enhance higher order thinking. These teachers have begun to find 
ways in which they can achieve their own and their school's goals of 
teaching for conceptual understanding, problem solving and reasoning in 
mathematics. 

We found that when teachers used alternative approaches to 
assessment, they also changed their teaching. Teachers increased their use 
of strategies that have been found by research to promote students' higher 
order thinking. They did activities that enhanced meaning and 
understanding, developed student autonomy and independence, and 
helped students learn problem solving strategies. Students' attitudes 
toward math improved. Students reported being more involved in group 
work, using calculators, making up their own problems, and other activities 
which reflect active rather than passive mathematics learning. 

Teachers provided feedback at the end of the project, outlining some 
of the abilities, knowledge, and skills that they had developed. They 
reported learning a variety of approaches for helping students to learn 
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basic mathematical facts and skills. The teachers said that they were more 
able to: 

• foresee students* problems in computational skills and emphasize steps 

to overcome these skill areas 

• develop a system of homework and quizzes to give an accurate 

description of their abilities 

• give different forms of assessment, having students show and 

explain their work 

• ask probing questions to encourage students to arrive at the correct 

answer 

• use questioning techniques which can help to recognize students' errors 

• use multiple approaches in assessing students' knowledge and skills 

• develop and use rubrics to score student work 

« feel comfortable in allowing students to use models and manipulatives 

Teachers also developed knowledge of strategies for teaching 
mathematical reasoning and problem solving. They reported feeling more 
capable in: 

• selecting open-ended problems that meet the level of experience of 

students, building problem difficulty systematically 

• using open ended problems to evaluate problem solving ability 

• looking for alternative methods and approaches 

• spending time discussing problem solving strategies 

• helping children not afraid to be afraid of thinking 
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• requiring students to engage daily in problen* solving and requiring 

them to demonstrate and communicate understandings in 
mathematics - orally and in writing 

• focusing on process rather than simply on an answer 

• being willing to try new ideas 

• using questioning techniques and encouraging the students to expand on 

answers and ideas 

• being able to follow the thought processes of all students 

• working with students one on one 

Although most of the teachers indicated that they still had much to 
learn about alternative approaches to assessment, they felt confident in 
trying further ideas, and expressed the desire to implement their ideas in 
the coming year. Many of them felt they could be even more successful if 
they were to implement these strategies at the beginning of the school 
year. 

Information about new approaches to mathematics assessment at the 
classroom level is critical for further progress in reforming mathematics 
education. If student performance on national, state, and local assessments 
is to meet the expectations of new standards, instruction and assessment 
must be closely aligned. This study provided evidence that inservice work 
on alternative assessment can pay dividends in helping mathematics 
teachers use approaches which enhance higher order thinking processes. 
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